Frederick Jelinek
   HOME

TheInfoList



OR:

Frederick Jelinek (18 November 1932 – 14 September 2010) was a
Czech-American Czech Americans ( cz, Čechoameričané), known in the 19th and early 20th century as Bohemian Americans, are citizens of the United States whose ancestry is wholly or partly originate from the Czech lands, a term which refers to the majority o ...
researcher in
information theory Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of information. The field was originally established by the works of Harry Nyquist a ...
,
automatic speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
, and
natural language processing Natural language processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to pro ...
. He is well known for his oft-quoted statement, "Every time I fire a linguist, the performance of the speech recognizer goes up". Jelinek was born in
Czechoslovakia , rue, Чеськословеньско, , yi, טשעכאסלאוואקיי, , common_name = Czechoslovakia , life_span = 1918–19391945–1992 , p1 = Austria-Hungary , image_p1 ...
before
World War II World War II or the Second World War, often abbreviated as WWII or WW2, was a world war that lasted from 1939 to 1945. It involved the vast majority of the world's countries—including all of the great powers—forming two opposin ...
and emigrated with his family to the United States in the
early years Early may refer to: History * The beginning or oldest part of a defined historical period, as opposed to middle or late periods, e.g.: ** Early Christianity ** Early modern Europe Places in the United States * Early, Iowa * Early, Texas * Early ...
of the communist regime. He studied engineering at the
Massachusetts Institute of Technology The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of the ...
and taught for 10 years at
Cornell University Cornell University is a private statutory land-grant research university based in Ithaca, New York. It is a member of the Ivy League. Founded in 1865 by Ezra Cornell and Andrew Dickson White, Cornell was founded with the intention to teach an ...
before accepting a job at
IBM Research IBM Research is the research and development division for IBM, an American multinational information technology company headquartered in Armonk, New York, with operations in over 170 countries. IBM Research is the largest industrial research org ...
. In 1961, he married Czech screenwriter
Milena Jelinek Milena Jelinek (Czech language, Czech: Milena Jelínková, née Tobolová; August 19, 1935 – April 15, 2020) was a Czech American screenwriter, playwright and teacher. She wrote the screenplay for the film ''Forgotten Light'', which was awarded ...
. At IBM, his team advanced approaches to computer speech recognition and machine translation. After IBM, he went to head the
Center for Language and Speech Processing Johns Hopkins University (Johns Hopkins, Hopkins, or JHU) is a private university, private research university in Baltimore, Maryland. Founded in 1876, Johns Hopkins is the oldest research university in the United States and in the western hem ...
at
Johns Hopkins University Johns Hopkins University (Johns Hopkins, Hopkins, or JHU) is a private university, private research university in Baltimore, Maryland. Founded in 1876, Johns Hopkins is the oldest research university in the United States and in the western hem ...
for 17 years, where he was still working on the day he died.


Personal life

Jelinek was born on November 18, 1932, as Bedřich Jelínek in
Kladno Kladno (; german: Kladen) is a city in the Central Bohemian Region of the Czech Republic. It has about 67,000 inhabitants. It is the largest city in the region and together with its adjacent suburban areas has a population of more than 110,000. ...
to Vilém and Trude Jelínek. His father was Jewish; his mother was born in Switzerland to Czech Catholic parents and had converted to Judaism. Jelínek senior, a dentist, had planned early to escape Nazi occupation and flee to England; he arranged for a passport, visa, and the shipping of his dentistry materials. The couple planned to send their son to an English
private school Private or privates may refer to: Music * " In Private", by Dusty Springfield from the 1990 album ''Reputation'' * Private (band), a Denmark-based band * "Private" (Ryōko Hirosue song), from the 1999 album ''Private'', written and also recorde ...
. However, Vilém decided to stay at the last minute and was eventually sent to the
Theresienstadt concentration camp Theresienstadt Ghetto was established by the Schutzstaffel, SS during World War II in the fortress town of Terezín, in the Protectorate of Bohemia and Moravia (German occupation of Czechoslovakia, German-occupied Czechoslovakia). Theresienstad ...
, where he died in 1945. The family was forced to move to
Prague Prague ( ; cs, Praha ; german: Prag, ; la, Praga) is the capital and largest city in the Czech Republic, and the historical capital of Bohemia. On the Vltava river, Prague is home to about 1.3 million people. The city has a temperate ...
in 1941, but Frederick, his sister and motherthanks to the latter's backgroundescaped the concentration camps. After the war, Jelinek entered in the gymnasium, despite having missed several years of schooling because education of Jewish children had been forbidden since 1942. His mother, anxious that her son should get a good education, made great efforts for their emigration,As he put it, "she didn't want to emulate my father's big mistake." especially when it became clear he would not be allowed to even attempt the graduation examination. His mother hoped her son would become a physician, but Jelinek dreamed of being a lawyer. He studied engineering in evening classes at the
City College of New York The City College of the City University of New York (also known as the City College of New York, or simply City College or CCNY) is a public university within the City University of New York (CUNY) system in New York City. Founded in 1847, Cit ...
and received stipends from the
National Committee for a Free Europe The National Committee for a Free Europe, later known as Free Europe Committee, was an anti-communist Central Intelligence Agency (CIA) front organization, founded on June 1, 1949, in New York City, which worked for the spreading of American influe ...
that allowed him to study at the
Massachusetts Institute of Technology The Massachusetts Institute of Technology (MIT) is a private land-grant research university in Cambridge, Massachusetts. Established in 1861, MIT has played a key role in the development of modern technology and science, and is one of the ...
. About his choice of specialty, he said: "Fortunately, to electrical engineering there belonged a discipline whose aim was not the construction of physical systems: the theory of information". He obtained his Ph.D. in 1962, with
Robert Fano Roberto Mario "Robert" Fano (11 November 1917 – 13 July 2016) was an Italian-American computer scientist and professor of electrical engineering and computer science at the Massachusetts Institute of Technology. He became a student and working ...
as his adviser. In 1957, Jelinek paid an unexpected visit to Prague. He had been in
Vienna en, Viennese , iso_code = AT-9 , registration_plate = W , postal_code_type = Postal code , postal_code = , timezone = CET , utc_offset = +1 , timezone_DST ...
and applied for a visa, hoping to see his former acquaintances again. He met with his old friend
Miloš Forman Jan Tomáš "Miloš" Forman (; ; 18 February 1932 – 13 April 2018) was a Czech and American film director, screenwriter, actor, and professor who rose to fame in his native Czechoslovakia before emigrating to the United States in 1968. Forman ...
, who introduced him to film student Milena Tobolováwhose screenplay had been the basis for the movie ''Easy Life'' (''Snadný život''). His flight back to the U.S. had a stopover in Munich, during which he called her to propose. Tobolová was considered a dissident and the authorities were not happy with her film. Jelinek asked for help from
Jerome Wiesner Jerome Bert Wiesner (May 30, 1915 – October 21, 1994) was a professor of electrical engineering, chosen by President John F. Kennedy as chairman of his Science Advisory Committee (PSAC). Educated at the University of Michigan, Wiesner was assoc ...
and
Cyrus Eaton Cyrus Stephen Eaton Sr. (December 27, 1883 – May 9, 1979) was a Canadian-American investment banker, businessman and philanthropist, with a career that spanned seventy years. For decades Eaton was one of the most powerful financiers in the ...
, the latter who lobbied
Nikita Khrushchev Nikita Sergeyevich Khrushchev (– 11 September 1971) was the First Secretary of the Communist Party of the Soviet Union from 1953 to 1964 and chairman of the country's Council of Ministers from 1958 to 1964. During his rule, Khrushchev s ...
. Following the inauguration of
John F. Kennedy John Fitzgerald Kennedy (May 29, 1917 – November 22, 1963), often referred to by his initials JFK and the nickname Jack, was an American politician who served as the 35th president of the United States from 1961 until his assassination i ...
, a group of Czech dissidents were allowed to emigrate in January 1961. Thanks to the lobbying, the future Milena Jelinek was one of them. After completing his graduate studies, Jelinek, who had developed an interest in
linguistics Linguistics is the scientific study of human language. It is called a scientific study because it entails a comprehensive, systematic, objective, and precise analysis of all aspects of language, particularly its nature and structure. Linguis ...
, had plans to work with
Charles F. Hockett Charles Francis Hockett (January 17, 1916 – November 3, 2000) was an American linguist who developed many influential ideas in American structuralism#Structuralism in linguistics, structuralist linguistics. He represents the post-Leonard Bloomfi ...
at
Cornell University Cornell University is a private statutory land-grant research university based in Ithaca, New York. It is a member of the Ivy League. Founded in 1865 by Ezra Cornell and Andrew Dickson White, Cornell was founded with the intention to teach an ...
. However these fell through and during the next ten years he continued to study information theory. Having previously worked at IBM during a sabbatical, he began full-time work there in 1972at first on leave for Cornell, but permanently from 1974. He remained there for over twenty years. Although at first he had been offered a regular research job, upon his arrival he learned that Josef Raviv had recently been promoted to head of the newly opened
IBM Haifa Research Laboratory IBM is a globally integrated enterprise operating in 170 countries. IBM's R&D history in Israel began in 1972 when Professor Josef Raviv established the IBM Israel Scientific Center in the Technion's Computer Science Building in Haifa. Today, ov ...
, and became head of the Continuous Speech Recognition group at the
Thomas J. Watson Research Center The Thomas J. Watson Research Center is the headquarters for IBM Research. The center comprises three sites, with its main laboratory in Yorktown Heights, New York, U.S., 38 miles (61 km) north of New York City, Albany, New York and with ...
. Despite his team's successes in this area, Jelinek's work remained little known in his home country because Czech scientists were not allowed to participate in key conferences. After the 1989 fall of communism, Jelinek helped establish scientific relationships, regularly visiting to lecture and helping to persuade IBM to establish a computing centre at
Charles University ) , image_name = Carolinum_Logo.svg , image_size = 200px , established = , type = Public, Ancient , budget = 8.9 billion CZK , rector = Milena Králíčková , faculty = 4,057 , administrative_staff = 4,026 , students = 51,438 , undergr ...
. In 1993, he retired from IBM and went to
Johns Hopkins University Johns Hopkins University (Johns Hopkins, Hopkins, or JHU) is a private university, private research university in Baltimore, Maryland. Founded in 1876, Johns Hopkins is the oldest research university in the United States and in the western hem ...
's Center for Language and Speech Processing, where he was director and Julian Sinclair Smith Professor of Electrical and Computer Engineering. He was still working there at the time of his death; Jelinek died of a heart attack at the close of an otherwise normal workday in mid-September 2010. He was survived by his wife, daughter and son, sister, stepsister, and three grandchildren, including Sophie Gold Jelinek.


Research and legacy

Information theory Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of information. The field was originally established by the works of Harry Nyquist a ...
was a fashionable scientific approach in the mid '50s. However, pioneer
Claude Shannon Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American people, American mathematician, electrical engineering, electrical engineer, and cryptography, cryptographer known as a "father of information theory". As a 21-year-o ...
wrote in 1956 that this trendiness was dangerous. He said, "Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems  ...  It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems."Quoted in Liberman (2010). During the next decade, a combination of factors shut down the application of information theory to
natural language processing Natural language processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to pro ...
(NLP) problemsin particular
machine translation Machine translation, sometimes referred to by the abbreviation MT (not to be confused with computer-aided translation, machine-aided human translation or interactive translation), is a sub-field of computational linguistics that investigates t ...
. One factor was the 1957 publication of
Noam Chomsky Avram Noam Chomsky (born December 7, 1928) is an American public intellectual: a linguist, philosopher, cognitive scientist, historian, social critic, and political activist. Sometimes called "the father of modern linguistics", Chomsky is ...
's ''
Syntactic Structures ''Syntactic Structures'' is an influential work in linguistics by American linguist Noam Chomsky, originally published in 1957. It is an elaboration of his teacher Zellig Harris's model of transformational generative grammar. A short monograph ...
,'' which stated, "probabilistic models give no insight into the basic problems of syntactic structure".Quoted in Young (2010). This accorded well with the philosophy of the
artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
research of the time, which promoted rule-based approaches. The other factor was the 1966
ALPAC ALPAC (Automatic Language Processing Advisory Committee) was a committee of seven scientists led by John R. Pierce, established in 1964 by the United States government in order to evaluate the progress in computational linguistics in general and m ...
report, which recommended that the government should stop funding research into machine translation. ALPAC chairman John Pierce later said that the field was filled with "mad inventors or untrustworthy engineers". He said that the underlying linguistic problems must be solved before attempts at NLP could be reasonably made. These elements essentially halted research in the field. Jelinek had begun to develop an interest in linguistics after the immigration of his wife, who initially enrolled in the MIT linguistics program with the help of
Roman Jakobson Roman Osipovich Jakobson (russian: Рома́н О́сипович Якобсо́н; October 11, 1896Kucera, Henry. 1983. "Roman Jakobson." ''Language: Journal of the Linguistic Society of America'' 59(4): 871–883. – July 18, The scope of research at IBM was considerably different from that of most other teams. According to
Mark Liberman Mark Yoffe Liberman is an American linguist. He has a dual appointment at the University of Pennsylvania, as Trustee Professor of Phonetics in the Department of Linguistics, and as a professor in the Department of Computer and Information Scienc ...
, "While elinekwas leading IBM's effort to solve the general dictation problem during the decade or so following 1972, most other U.S. companies and academic researchers were working on very limited problems  ...  or were staying out of the field entirely". Jelinek regarded
speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
as an information theory problema
noisy channel In information theory, the noisy-channel coding theorem (sometimes Shannon's theorem or Shannon's limit), establishes that for any given degree of noise contamination of a communication channel, it is possible to communicate discrete data (dig ...
, in this case the acoustic signalwhich some observers considered a daring approach. The concept of
perplexity In information theory, perplexity is a measurement of how well a probability distribution or probability model predicts a sample. It may be used to compare probability models. A low perplexity indicates the probability distribution is good at ...
was introduced in their first model, New Raleigh Grammar, which was published in 1976 as the paper "Continuous Speech Recognition by Statistical Methods" in the journal ''
Proceedings of the IEEE The ''Proceedings of the IEEE'' is a monthly peer-reviewed scientific journal published by the Institute of Electrical and Electronics Engineers (IEEE). The journal focuses on electrical engineering and computer science. According to the ''Journa ...
''. According to Young, the basic noisy channel approach "reduced the speech recognition problem to one of producing two statistical models". Whereas New Raleigh Grammar was a
hidden Markov model A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process — call it X — with unobservable ("''hidden''") states. As part of the definition, HMM requires that there be an ob ...
, their next model, called Tangora, was broader and involved
n-gram In the fields of computational linguistics and probability, an ''n''-gram (sometimes also called Q-gram) is a contiguous sequence of ''n'' items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or b ...
s, specifically trigrams. Even though "it was obvious to everyone that this model was hopelessly impoverished", it was not improved upon until Jelinek presented another paper in 1999. The same trigram approach was applied to
phones A telephone is a telecommunications device that permits two or more users to conduct a conversation when they are too far apart to be easily heard directly. A telephone converts sound, typically and most efficiently the human voice, into ele ...
in single words. Although the identification of
parts of speech In grammar, a part of speech or part-of-speech (abbreviated as POS or PoS, also known as word class or grammatical category) is a category of words (or, more generally, of lexical items) that have similar grammatical properties. Words that are assi ...
turned out not to be very useful for speech recognition, tagging methods developed during these projects are now used in various NLP applications. The incremental research techniques developed at IBM eventually became dominant in the field after
DARPA The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military. Originally known as the Adv ...
, in the mid-80s, returned to NLP research and imposed that
methodology In its most common sense, methodology is the study of research methods. However, the term can also refer to the methods themselves or to the philosophical discussion of associated background assumptions. A method is a structured procedure for bri ...
to participating teams, shared common goals, data, and precise evaluation metrics. The Continuous Speech Recognition Group's research, which required large amounts of data to train the algorithms, eventually led to the creation of the
Linguistic Data Consortium The Linguistic Data Consortium is an open consortium of universities, companies and government research laboratories. It creates, collects and distributes speech and text databases, lexicons, and other resources for linguistics research and developm ...
. In the 1980s, although the broader problem of speech recognition remained unsolved, they sought to apply the methods developed to other problems; machine translation and stock value prediction were both seen as options. A group of IBM researchers went on to work for
Renaissance Technologies Renaissance Technologies LLC, also known as RenTech or RenTec, is an American hedge fund based in East Setauket, New York, on Long Island, which specializes in systematic trading using quantitative models derived from mathematical and statisti ...
. Jelinek wrote, "The performance of the Renaissance fund is legendary, but I have no idea whether any methods we pioneered at IBM have ever been used. My former colleagues will not tell me: theirs is a very hush-hush operation!" Methods very similar to those developed for achieving speech recognition are at the base of most machine translation systems in use today. Observers have said that Pierce's paradigm, according to which engineering achievements in this area would be built on scientific progress, has been inverted, with the achievements in engineering being at the base of a number of scientific findings. Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM. He received the Society Award for "outstanding technical contributions and leadership" from the
IEEE Signal Processing Society The IEEE Signal Processing Society (IEEE SPS) is one of the nearly 40 technical societies of the Institute of Electrical and Electronics Engineers (IEEE) and the first one created. Its mission is to "advance and disseminate state-of-the-art scien ...
for 1997, and the
ESCA The anglerfish are fish of the teleost order Lophiiformes (). They are bony fish named for their characteristic mode of predation, in which a modified luminescent fin ray (the esca or illicium) acts as a lure for other fish. The luminescence co ...
Medal for Scientific Achievement in 1999. He was a recipient of an IEEE Third Millennium Medal in 2000, the
European Language Resources Association The European Language Resources Association (ELRA) is a not-for-profit organisation established under the law of the Grand Duchy of Luxembourg. Its seat is in Luxembourg and its headquarters is in Paris, France. Activities Since its founding in ...
's first Antonio Zampolli Prize in 2004, the 2005 James L. Flanagan Speech and Audio Processing Award, and the 2009 Lifetime Achievement Award from the
Association for Computational Linguistics The Association for Computational Linguistics (ACL) is a scientific and professional organization for people working on natural language processing. Its namesake conference is one of the primary high impact conferences for natural language proces ...
. He received an ''honoris causa'' Ph.D. from
Charles University ) , image_name = Carolinum_Logo.svg , image_size = 200px , established = , type = Public, Ancient , budget = 8.9 billion CZK , rector = Milena Králíčková , faculty = 4,057 , administrative_staff = 4,026 , students = 51,438 , undergr ...
in 2001, was elected to the
National Academy of Engineering The National Academy of Engineering (NAE) is an American nonprofit, non-governmental organization. The National Academy of Engineering is part of the National Academies of Sciences, Engineering, and Medicine, along with the National Academy ...
in 2006 and was made one of twelve inaugural fellows of the
International Speech Communication Association The International Speech Communication Association (ISCA) is a non-profit organization and one of the two main professional associations for speech communication science and technology, the other association being the IEEE Signal Processing Society ...
in 2008.


Selected publications

*Jelinek, Frederick (1968). ''Probabilistic Information Theory: Discrete and memoryless models''. McGraw-Hill series in systems science. New York: McGraw-Hill. 689p. (review) *———————- (1969). "Fast sequential decoding algorithm using a stack". ''IBM Journal of Research and Development'' 13(6):675–685. . *———————- (1969). "Tree encoding of memoryless time-discrete sources with a fidelity criterion". ''IEEE Transactions on Information Theory'' 15(5):584–590. . (received 1971 "Best Paper" award) *Bahl, Lalit R.; John Cocke, Frederick Jelinek, Josef Raviv (1974). "Optimal decoding of linear codes for minimizing symbol error rate". ''IEEE Transactions on Information Theory'' 20(2):284–287. . (received Information Theory Society Golden Jubilee paper award) *———————- (1976). "Continuous speech recognition by statistical methods". ''Proceedings of the IEEE'' 64(4):532–556. . *Brown, P.; J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R, Mercer and P. Roossin (1988)
"A statistical approach to language translation"
. In Dénes Vargha, ed. ''Coling 88: Proceedings of the 12th conference on Computational linguistics, volume 1''. Budapest: John Von Neumann society for computing sciences. pp. 71–76. . . *———————- (1990). "Self-Organized Language Modeling for Speech Recognition". In Alex Waibel & Kai-Fu Lee, eds. ''Readings in speech recognition''. San Mateo: Morgan Kaufmann. 629p. . *———————-; John D. Lafferty and Robert L. Mercer. (1990) "Basic methods of probabilistic context free grammars". Technical Report RC 16374 (72684), IBM. **Reprinted in Laface, Pietro; Renato De Mori (1992). ''Speech Recognition and Understanding: Recent advances, trends, and applications''. NATO ASI series. Series F, Computer and systems sciences, 75. New York: Springer-Verlag. pp. 345–360. . *———————- (1997)
''Statistical Methods for Speech Recognition''
Cambridge, Mass.: MIT Press. 283p.
(review)(review 2)
* Chelba, Ciprian; Frederick Jelinek (2000). "Structured Language Modeling". ''Computer Speech & Language'' 14(4):283–332. (received 2002 "Best Paper" award). **Expanded version of a presentation at NLDB'99. Klagenfurt, Austria, June 17–19, 1999 (). * Xu, Peng; Ahmad Emami and Frederick Jelinek (2003).
Training Connectionist Models for the Structured Language Model
. In Michael Collins and Mark Steedman, eds. ''EMNLP '03 Proceedings of the 2003 conference on Empirical methods in natural language processing''. East Stroudsburg, Penn.: Association for Computational Linguistics. pp. 160–167. . . (won "best paper" award)


References

; Notes ; References


External links


Institutional page
at Johns Hopkins university {{DEFAULTSORT:Jelinek, Frederick 1932 births 2010 deaths American people of Bohemian descent American people of Czech-Jewish descent Statistical natural language processing Czechoslovak emigrants to the United States Cornell University faculty Harvard University faculty Johns Hopkins University faculty MIT School of Engineering alumni IBM Research computer scientists IBM employees Members of the United States National Academy of Engineering People from Kladno Speech processing researchers Natural language processing researchers